Merged
Conversation
Add a new skill to agentv-dev for reviewing AI plugin PRs across three dimensions: skill quality, eval correctness, and workflow architecture. Includes: - SKILL.md with 5-step review process - references/eval-checklist.md for eval file review - references/workflow-checklist.md for OpenSpec-based workflow review - 8-test eval covering missing evals, naming, assertions, file paths, repeated inputs, hard gates, factual contradictions, and command refs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Deploying agentv with
|
| Latest commit: |
2be87c9
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://6b7e1cee.agentv.pages.dev |
| Branch Preview URL: | https://feat-agentv-plugin-review.agentv.pages.dev |
Rewrite eval to use pi-cli target with a mock workspace containing a deploy-auto plugin with planted issues: missing eval, wrong extensions, no assertions, relative paths, repeated inputs, missing hard gates, factual contradiction, nonexistent command reference, and hardcoded paths. Uses shorthand input, skill-trigger assertions, and workspace template instead of loading SKILL.md directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The pi-coding-agent provider is configured as the 'default' target in .agentv/targets.yaml. pi-cli is not a valid target name. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pi-cli as a named target in .agentv/targets.yaml using the pi-coding-agent provider. Update the plugin-review eval to use it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add references/skill-quality-checklist.md covering CSO (Claude Search Optimization), description anti-patterns, content quality, file org, flowchart usage, and discipline-enforcing skill checks. Update SKILL.md Step 2 to reference the checklist and integrate key CSO principles (description must not summarize workflow, etc). Remove plugin.json from review table (handled by CI). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ugin review skills Move agent-plugin-review from agentv-dev to new agentic-architect plugin. Add agent-architecture-design skill with species taxonomy (Coding Harness, Dark Factory, Auto Research, Orchestration) and workflow patterns from OpenSpec, Superpowers, and Compound Engineering. New plugin structure: - agent-architecture-design: diagnose problem, select species, design workflow - agent-plugin-review: review skills, evals, and workflow architecture Register in both .claude-plugin and .github marketplace.json. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- pi-cli target now uses provider: pi-cli (matching its name) - pi-coding-agent target added as explicit alias matching default config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ic design patterns" Rename species-taxonomy.md to agentic-design-patterns.md. Replace all "Species A/B/C/D" labels with standard terms: Single-Agent Iterative Loop, Autonomous Pipeline, Optimization Loop, Multi-Agent System. Based on Andrew Ng's agentic design patterns framework and industry conventions (cognitive architectures, agentic workflows). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These evals validate the agentv repo itself, not architecture patterns. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- agentv-eval-review: new skill in agentv-dev with lint_eval.py script for .eval.yaml naming, file paths, assertions, prose detection, repeated inputs - agent-plugin-review: add lint_plugin.py script for frontmatter, hardcoded paths, version printing, missing evals, referenced files, command refs - Remove eval-checklist.md from agent-plugin-review (handled by agentv-eval-review) - Both SKILLs are now minimal wrappers: run script first, then LLM judgment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Address remaining LLM-judgment gaps from PR #324 review: - Internal self-consistency (skill contradicting itself) - Cross-file consistency (filenames/tools matching across skills and evals) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AGENTS.md or instruction files with heavy TRIGGER/ACTION routing tables indicate skill descriptions aren't enabling auto-discovery. Good descriptions make manual routing unnecessary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skills that depend on MCP servers, external repos, specific directory layouts, or services should declare these explicitly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rences External dependencies is not a best practice per agentskills.io — the compatibility field and gotchas sections handle this. The real issue (hardcoded paths) is already caught by lint_plugin.py. Add agentskills.io specification, best practices, and description optimization as external references for discoverable guidance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Check relative markdown links on push to main and PRs using lychee. Offline mode only (local links, not HTTP URLs). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- apps/cli/README.md: add ../../ prefix for repo-root references - examples/showcase/offline-grader-benchmark/README.md: fix CLAUDE.md depth - packages/eval/README.md: agentv-eval-builder → agentv-eval-writer Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
apps/examples is a symlink to ../examples which causes lychee to resolve relative paths incorrectly (../../../CLAUDE.md resolves to apps/CLAUDE.md instead of repo root CLAUDE.md). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unused symlink to ../examples that confused lychee link checker. examples/ is accessible directly at repo root. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
apps/cli/README.md is published to npm and was a stale copy of the root README with diverging relative links. Replace with a symlink so it stays in sync automatically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
agentv-plugin-reviewskill to theagentv-devplugin for reviewing AI plugin PRsFiles
Skill:
plugins/agentv-dev/skills/agentv-plugin-review/SKILL.md— 5-step review processplugins/agentv-dev/skills/agentv-plugin-review/references/eval-checklist.md— Eval file review checklistplugins/agentv-dev/skills/agentv-plugin-review/references/workflow-checklist.md— Workflow architecture checklistEval:
evals/agentv-plugin-review/agentv-plugin-review.eval.yaml— 8 test casesTest plan
🤖 Generated with Claude Code